The increased use of computers in many applications has drawn increasing focus on improving the man-machine interface. It is the desire of many applications to locate the face of the user, then to process it to robustly identify the person (for entitlement benefits recipients, for national border crossings or secure area entry verification, as a replacement for the ubiquitous PIN numbers, etc.). The algorithms for facial recognition have dramatically improved in recent years and are now sufficiently robust for many applications. The weak part of the system is the face detection and location front-end system. Other applications for facial imaging beyond identification are growing in interest, in particular perceptual computing, such as discerning a reaction or emotion from a user's face. This would enable computer-driven systems to be more responsive, like a human. Again, these algorithms will be limited by the weaknesses in face detection and location.
When flash illumination is used during the capture of an image that contains sizable human faces, the pupils of people sometimes appear red because the light is partially absorbed by capillaries in the retina. As illustrated in FIG. 1, the light rays 70 from the flash illumination source 12 enter the eye 2 through the eye lens 3, and form an image 12a of the illumination source 12 on retina 4. The eye-defect in captured images, known as the "red-eye effect" is mostly seen with human eyes. In case animals are captured, the eye-defect will show a bright green or yellow color. Animal eyes are generally more difficult to detect for pattern recognition algorithms due to the large variations in animal facial structure, complexion, hair and structure of the eyes itself.
Referring now to FIG. 2, the light rays 80 reflected from the retina 4 exit the eye 2 through the eye lens 3, and finally enter the camera lens 5. If the camera lens 5 is placed close to the illumination source 12, the red-eye effect will be maximized. In other words, the amount of red-eye or eye-defect being observed increases as the illumination source 12 gets closer to an optical axis 22 defined by the camera lens 5 (see also FIG. 3).
The general technique for red-eye reduction in cameras has been to impact two parameters: (a) Reduce the pupil diameter of the subject, for example by emitting a series of small pre-flashes prior to capturing the desired image with full illumination; and, (b) Increase the flash to lens separation, so that the illumination impinging on the subjects eyes is reflected at an angle that misses the taking lens.
A number of U.S. patents reflect the art prior to this invention. Each will be discussed:
U.S. Pat. No. 5,570,434, issued Oct. 29, 1996 to Badique, entitled "Circuit Arrangement For Recognizing A Human Face", describes a circuit for recognizing a human face in a sequence of video images. The circuit includes the steps of subtracting two consecutive image frames and using the areas of difference to determine if there is a moving object. The difference map receives further post-processing, culminating in a convolution to determine if a head and shoulders feature set is present in the scene.
U.S. Pat. No. 5,680,481, issued Oct. 21, 1997 to Prasad et al., entitled "Facial Feature Extraction Method and Apparatus for a Neural Network Acoustic and Visual Speech Recognition System", describes another approach to facial feature identification and extraction. In this patent, the variation in gray scale tones from a dull frontal face image are used to locate the eyes and mouth by thresholding the pixel values and finding centroids of the three areas.
Many other prior art documents deal more specifically with eye or gaze tracking, or iris recognition methods or, face matching methods.
U.S. Pat. No. 5,325,133, issued Jun. 28, 1994 to Adachi, entitled "Device for Measuring a Retina Reflected Light Amount and a Gaze Detecting Apparatus Using the Same" describes a device for measuring reflected light from a retina and detecting the direction in which the user is looking with an apparatus. This apparatus directs three sources of infrared emission, located at three different positions, toward a user's eyes. The system locates the pupils by pattern recognition or by red-eye ("detecting the frequency components corresponding to hemoglobin which is abundantly contained in the light reflected by the retina"). It then processes the retinal reflection based on the displacement angles to determine where the user's gaze is directed. Thus, the apparatus functionality is keyed to computing the gaze angle from the three dimensional angular measurements, instead of locating eyes. This art does not mention the issue of spacing between emission source and image capture device. It teaches away from the current invention by its dependence on measuring angular separation between emission source and image pick-up source for a minimum of three emitter/detector pairs.
U.S. Pat. No. 5,231,674 issued Jul. 27, 1993 to Cleveland et al., entitled "Eye Tracking Method and Apparatus" describes a system to determine the direction that a person is gazing, determining the point at which he is gazing, or measuring the motion of his eye. The outputs of this apparatus are locations of eye features such as edge coordinates between the pupil and iris of the eye and of the center coordinates of light reflections off the cornea of the eye. An image intensity profile taken through a cross section of the eye iris, pupil (illuminated using the bright-eye effect and the corneal reflection is extracted knowing the size of the eye, inferred from the distance between the eye and the cameras). Then, the extracted profile is smoothed. The peak region (defined by a set of thresholds) near the center of the smoothed profile is detected as the corneal region. Typically the relative intensities of the iris and the pupil are not sufficiently different to make the image processing easy. Therefore, the bright-eye effect is used in this method and apparatus to increase the contrast ratio between the pupil and its surrounding iris. Consequently, the contrast ratio between the iris and the bright pupil in the camera image can be made significantly greater than the contrast ratio between the iris and a dark pupil. With the increased contrast ratio, image processing algorithms can locate the pupil edges and center more reliably and accurately for the particular eye gaze-tracking application. It would be noted that this method and apparatus assumes known eye location and eye size (on the camera optical axis and at a fixed distance) and is designed to track the eye gaze for speech impaired and physically handicapped persons. Moreover, this method and apparatus uses temporal changes in monochrome images and, thus, no color information.
U.S. Pat. No. 5,432,863 issued Jul. 11, 1995 to Benati et al., entitled "Automated Detection and Correction of Eye Color Defects Due to Flash Illumination" describes a means to automatically detect and correct red-eye defects due to flash illumination in still images. The method includes means for determining whether a still image artifact is truly a red-eye problem based on shape, coloration, and brightness.